The present invention relates to modification of video data, and particularly to that for synthesizing identification data thereto for identifying the video data.
Illegal reproduction of digital video data is all actual problem. For preventing the illegal reproduction, there is proposed a playback system wherein digital video data are encoded into cipher data which can be played back only by video players equipped with a proper decipher key. However, even the ciphered video data, the illegal reproduction cannot be prevented once they are deciphered.
Hence, embedding special information into the digital video data themselves is attempted for preventing the illegal usage and reproduction. In the following paragraphs, data representing this special information will be called the electronic watermark data.
The electronic watermark data can be classified into two kinds, that is, visible electronic watermark data and invisible electronic watermark data.
An example of the visible electronic watermark data is disclosed in a Japanese patent application laid open as a Provisional Publication No. 241403/""96. In the example, for synthesizing the electronic watermark data into original picture data, the brightness values of original pixels data corresponding to opaque pixel of the electronic watermark are modified leaving color differential components thereof unchanged. When modifying the brightness values, a scaling factor to be applied for the modification may be determined according to a color component, random number, pixel value of the electronic water mark, or others.
Thus, the visible electronic watermark modifies a picture so that the modification may be sensed visually by comparing, or even without comparing, to its original, by synthesizing special characters or marks into the picture. Hence, the visible electronic watermark is effective in the prevention of illegal use to the observer. However, degradation of picture quality is, more or less, unavoidable.
On the other hand, the invisible electronic watermark is the watermark unable to be sensed visually, being embedded within image data and does not degrade the picture quality.
By embedding, into a video program, special information for identifying its licensee, for example, as the invisible electronic watermark, identification of the licensee becomes possible by extracting the watermark data from illegal copies of the video program. It becomes also possible, by embedding reproduction-prohibitive information in the program, to provide an alarm or restrict a user to make duplication by a VTR (Video Tape Recorder), by operating a copy prevention mechanism therein, for example, when the reproduction-prohibitive information is detected by the VTR.
The invisible electronic watermark data may be embedded into a part of image data which gives little affect to the picture quality, such as into LSB (Least Significant Bit) of each pixel value.
However, when the watermark is embedded into the LSB, it can be easily eliminated by way of a low-pass filter. Further, tie image data compression generally operates by reducing data amount by omitting data parts having little affect to the picture quality. That means the watermark embedded there is also eliminated by the image data compression.
As above described, there has been a tradeoff between the picture quality and the traceability of the watermark in the invisible electronic watermark.
For evading this tradeoff, a method of embedding the watermark data into frequency spectrum data of an image is proposed (in p. 13 of the NIKKEI ELECTRONICS, no. 660, Apr. 22, 1996). In this method, which will be called the frequency-domain method in the following paragraphs, being embedded into frequency components, the watermark data have sufficient durability to the image data processing such as data compression or filtering, and further, interference between different watermark data is also prevented, making it difficult to break the watermark data without giving serious affects throughout the picture.
In the frequency-domain method, the electronic watermark is embedded as follows, for example.
Original image data are transferred into frequency components by way of the DCT (Discrete Cosine Transform). Selecting n components f(1), f(2), . . . , f(n) each having higher n values among the frequency components thus obtained, an electronic watermark data set w(1), w(2), . . . , w(n) is prepared so as to accord to a normal distribution having a mean value 0 and a variance 1. Then, F(i)=f(i)+xcex1|f(i)|xc3x97w(i) is calculated for each i (i=1, 2, . . . , n), xcex1 being a scale factor. From the frequency components wherein each f(i) is replaced with F(i), the image data including the electronic watermark are obtained.
For detecting the electronic watermark, following processes are performed, for example, on condition that the original image data and the probable watermark data set w(i) (i=1, 2, . . . , n) are known.
First, frequency components F(1) to F(n) corresponding to those f(1) to f(n) of the original image as above defined are extracted from image data wherein the electronic watermark is considered to be embedded. Then, each i-th component W(i) of a probable data set vector W=(W(1), W(2), . . . W(n)) is calculated as W(i)=(F(i)xe2x88x92f(i))/f(i). When a static resemblance C of the probable data set vector W to the watermark data set vector w=(w(1),w(2), . . . , w(n)) obtained from their normalized inner product represented by C=Wxc2x7xcfx89/(|W|xc2x7|xcfx89|) is larger than a certain value, the concerning electronic watermark is determined to be embedded in the object image data.
Thus, the author of the original image can effectively verify illegal copies by detecting the electronic watermark embedded by the frequency-domain method in the digital image data according to each licensee.
FIG. 6 is a block diagram illustrating a data modifier to be applied to the prior art for embedding the electronic watermark data by way of the frequency-domain method. Original image data 601 are converted into frequency components by a DCT processor 602, which are modified by a watermark embedder 603 according to a watermark data set 604 and converted by an inverse DCT processor 605 into time-domain data to be output as the watermarked image data 606.
It is proposed to apply the above frequency-domain method to MPEG (Moving Picture Expert Group) video data. FIG. 7 is a block diagram illustrating the basic configuration of a video-data encoder 700 for generating the MPEG video data, wherein are comprised a DCT processor 101 for converting an original video data stream 110 into frequency-domain data, a quantizer 104 for thinning out negligible data therefrom and a variable-length coder 105 for encoding the quantized data into the MPEG data stream 120. As can be seen from FIGS. 6 and 7, the DCT should be performed two times when the data modifier of FIG. 6 is directly applied in the video-data encoder 700 of FIG. 7. The encoding procedure of the video data into compressed data charges a heavy load to the encoder needing a large amount of operation. Therefore, operational load for embedding the electronic watermark is expected, without saying, to be as small as possible.
Therefore, a primary object of the present invention is to provide a method of and an apparatus for embedding watermark data according to the frequency-domain method efficiently into video data.
In order to achieve the object, a video-data encoder of the invention for performing encoding of an original video data into a compressed data stream having a multi-layer structure comprises;
means for transforming the original video data into a sequence of processing units of frequency domain data, said processing units being components of a predetermined layer of the multi-layer structure;
means for embedding predetermined watermark data into at least one unit of the sequence of processing units; and
means for generating the compressed data stream by processing the sequence of processing units.
Therefore, the data transformation of the original video data into the frequency domain data for embedding the watermark data according to the frequency-domain method can be performed efficiently without any additional process, by exploiting the data transformation for encoding the original video data into the compressed data stream.
Further, in an embodiment of the invention, where the encoding is performed according to the MPEG standard, the watermark data are embedded into one or some of macro-blocks determined referring to a criterion defined in connection with anyone of a slice, a picture, a field, a frame, or a GOP.
Therefore, additional load of the video-data encoder for embedding the watermark data can be still suppressed.